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Abstract. This paper presents simulation-based relations for proba- 
bilistic game structures. The first relation is called probabilistic alter- 
nating simulation, and the second called probabilistic alternating for- 
ward simulation, following the naming convention of Segala and Lynch. 
We study these relations with respect to the preservation of properties 
specified in probabilistic alternating-time temporal logic. 

1 Introduction 

Simulation relations [Mil89] have proved to be useful for comparing the behavior 
of concurrent systems, which can be formally interpreted as labeled transition 
systems. The study of logic characterization of simulation is to build its connec- 
tion to a modal or temporal logic which can be used to formulate some interest- 
ing properties. Soundness of logic characterization requires simulation preserve 
the satisfaction of logic formulas, while completeness shows the relation has the 
same strength as the logic. Intuitively, the fact that one state si simulates an- 
other state S2 can be used to establish the relation that any possible behavior 
of si is also possible on S2- Thus it can preserve certain desirable properties^ 
formulated in temporal logics like CTL [Eme90]. Simulation relations have set 
up the foundations for constructing correct abstractions. 

Related work. Segala and Lynch [SL95] extend the classical notions of simula- 
tion for probabilistic automata [Seg95b], a general extension of labeled transi- 
tion systems which admits both probabilistic and nondeterministic behaviors. 
Their main idea is to relate probability distributions over states, instead of re- 
lating individual states. They show soundness of the logical characterization 
of probabilistic simulation, which preserves probabilistic CTL formulas [Han94] 
without negation and existential quantification. Segala introduces the notion 
of probabilistic forward simulation, which relates states to probability distribu- 
tions over states and is sound and complete for trace distribution precongru- 
ence [Seg95a,LSV07]. Logic characterization of strong and weak probabilistic 
bisimulation has been studied in [DGJP02,PS07]. 

Alur, Henzinger and Kupferman [AHK97,AHK02] define ATL (alternating- 
time temporal logic) to generalize CTL for game structures by requiring each 



^ For example, safety properties stating "nothing bad can happen" . 



path quantifier to be parametrized with a set of agents. Game structures are 
more general than LTS, in the sense that they allow both collaborative and 
adversarial behaviors of individual agents in a system, and ATL can be used 
to express properties like "a set of agents can enforce a specific outcome of the 
system" . Alternating refinement relations, in particular alternating simulation, 
are introduced later in [AHKV98]. Alternating simulation is a natural game- 
theoretic interpretation of the classical simulation in two-player games. Logic 
characterization of this simulation concentrates on a subset of ATL* formulas 
where negations are only allowed at proposition level and all path quantifiers 
are parametrized by a prefixed set of agents A. This sublogic of ATL* contains 
all formulas expressing the properties the agents in A can enforce no matter 
what the other agents do. Alur et al. [AHKV98] have proved both soundness 
and completeness of the logic characterization. 

Our contribution. In this apper, we introduce two notions of simulation for prob- 
abilistic game structures - probabilistic alternating simulation and forward sim- 
ulation, following the aforementioned results [Seg95a,SL95,AHKV98]. We prove 
the soundness of logical characterization of probabilistic alternating simulation 
relations, by showing that they preserve a fragment of a probabilistic extension 
of ATL. 

Structure of the paper. The rest of the paper is organized as follows. Wc briefly 
explain some basic notations that are used throughout the paper in Sect. 2. 
Sect. 3 introduces the notion of probabilistic game structures and the defini- 
tion of probabilistic executions. In Sect. 4 we present PATL an extension of the 
alternating-time temporal logic [AIIK02] for probabilistic systems, and roughly 
discuss its model checking problem. We define probabilistic alternating simula- 
tion and forward simulation in Sect. 5, and show their soundness for preserving 
properties specified in PATL in Sect. 6. Probabilistic alternating bisimulation is 
shortly discussed in Sect. 7. We conclude the paper with some future research 
topics in Sect. 8. 

2 Preliminaries 

This section contains basic notions that are used in the technical part. Let 5 be a 
set. A discrete probabilistic distribution A over 5 is a function of type S [0, 1], 
satisfying XlseS ^('^) ^ ^- write I?(S') for the set of all such distributions. 
For a set S' C S, define A{S') = YliseS' ^{^)- Given two distributions Ai,A2 
and p e [0, 1], Al (BpA2 is a function of type S [0, 1] defined as Ai(BpA2{s) = 
p- Ai{s) + {l—p) ■ A2{s) for all s E S. Obviously, Ai(BpA2 is also a distribution. We 
further extend this notion by combining a set of distributions {Ai}i^j ordered 
by an indexed set {pi}i^i into a distribution J2iei Pi^i' where Pi G [0, 1] for all 
i G I and X^ig/K = 1. s is called a point distribution satisfying s(s) ~ 1 and 
s{t) = for all t ^ s. Let A G ^{S), write [Z\] for the support of A as the set 
{sgS\ A(s) > 0}. 



Let S = Si X S2 X ■ ■ ■ y- Sn, then s G S is a. vector of length n. We may 
also write s = (si, S2, . . . , .s,i), with s{i) = Si € Si. Given a finite sequence 
a = S1S2 . . . Sn G S* , write last{a) for s„. Let S" C S, then q | S" is a subsequence 
of a with exactly the elements not in S' removed. Given L C S*, write L \ S' 
for the set {{a \ S') \ a G L}. 

3 Probabilistic Game Structures 

Assume a set of players E = {1,2, . . . , k}. A probabilistic game structure (PGS) 

Q is defined as a tuple {S, sq, C, Act, S), where 

— S' is a finite set of states, with sq the initial state, 

— Act = Acti X Act2 X • • • X Actk is a set of joint actions, where Actj is the set 
of actions for player i = 1, . . . , k, 

— C: S ^ 2''^°P is the labelUng function, 

— 6 : S X Act — )• 'D{S) is a transition function. 

A play p is a (finite or infinite) sequence soai,sia2S2 . . ., such that a; G Act and 
S{si-i,ai){si) > for all i. Write \p\ for the length of a run p, which is the number 
of transitions in p, and \p\ = 00 if p is infinite. We write p{i) for the i-th state in p 
starting from 0, and p[i, j] for the subsequence starting from i-th state and ending 
at the j-th state, provided < i < j < \p\. Note that the players choose their 
next moves simultaneously, but their moves may or may not be cooperative. 
If on state s each player i performs action a^, then S{s, {ai, a2, . . . ak)) is the 
distribution for the next reachable states. In the following discussion, we fix a 
probabilistic game structure Q. 

We assume that the transition relation is total on the set Act. Note that this 
does not pose any limitation on the expressiveness of the model. If an action 
c e Actj of player i is not supposed to be enabled on state s for player i, we 
may find another action c' G Act^ and define c to have the same effect as c' 
on s. Since player i knows the current state, he also knows the set of actions 
available to him, so that as a rational player he will not choose actions that are 
not enabled. This allows such models to express systems in which on some states 
the available (joint) actions are proper subsets of Act.^ We may even disable a 
particular player on a state. A player i is disabled on s if S{s, a) = 5{s, a') for all 
action vectors a, a' G Act satisfying a{j) = a'{j) for all ) 7^ i. A PGS is turn- 
based if all but one player is disabled on s for all s G S. A probabilistic game 
structure can be regarded as a generalization of a concurrent game structure 
of [AHK02]. From a state ,s G S, each player i may choose an action from Actj 
and together they resolve the nondeterminism. On the other hand, a PGS is 
more stratified on external actions than some of the existing models.^ 

^ In the literature some authors encode available actions for player i as a function of 

typeS'^2*'=*-\{0}. 

^ For example, a one-player PGS resembles a reactive system of [vGSS95], and a two- 
player turn-based PGS (assuming they alternately act) loosely simulates a simple 
probabilistic automaton [Seg95b], in the way that one player performs external ac- 
tions and the other resolves nondeterminism after the previous player's move is done. 



A strategy of a player z G 17 is a function of type — >■ I?(Acti). We 
write nf for the set of strategies of player i in Q.^ A play p is compatible 
with an i-strategy TTj, if afc(i) € [7ri(p[0, A; — l]\S)'] for all fc < \p\. Given a 
vector of strategies tt € ilf x ilj' x • • • x ^?^|^|, a run p is compatible with tt if 
ttkii) e [7r(i)(p[0, fc- Ijl^)] for all A; < |p| and i = 1, . . . ,k. Write g{n, s) for the 
set of infinite plays compatible with every strategy in tt starting from s € S, and 
g* (tt, s) the set of finite plays in Q that are compatible with tt starting from s. 

The set of finite plays compatible to a strategy vector tt is also called a set 
of cones [Scg95b], with each finite play a representing the set of infinite plays 
prefixed by a. Given a state Sq G S, we can derive the probability for every mem- 
ber in S+ compatible with tt, by recursively defining a function Pr5(^ from 

to [0, 1] as follows. This function Prg(T^_s^) can be further generalized as the 
probability measure to the cr-field ■Fg,-K,so ^ G{'!^, so) which is a unique extension 
from the set of cones Q* (tt, s) closed by countable union and complementation, 
in a way similar to [Seg95b] : 

- Prg^^^s^){a-s) = Prg(^^^,^){ay5{last{a), (7r(l)(a), 7r(2)(Q!), . . . ,7r(k)(a)))(s), 

where 6{s, {Ai,A2, . . . , Ay^)) is a distribution over states derived from 5 and the 
vector of action distributions defined by 

6is,{Ai,...,Ay,)) = J2 Ai{ai)-...-A^{ai,)-5{s,{ai,...,a^)). 

ie{l,...,k},aie[Zii] 

Given ACE, sometimes we write ■k{A) for a vector of \A\ strategies {TTijig^, 
and n{A) for the set of all such strategy vectors. Write A for E \ A. Given 
A n A' = 0, strategy vectors tt € n{A) and tt' € n{A'), tt U tt' is the vector of 
strategies {ni}i^A U {nj}j^A' that combines tt and tt'. 

We also define strategies of fi/riMe depth by restricting the size; of their do- 
mains, by writing tt € nf'" as a level-n strategy, i.e., tt is a function from traces 
of states with length up to n (i.e., the set Ume{i 2 «} '^™) T>{kcti). Given a 
set of strategies {TTijig/ of the same domain, and with ^^^jPi =■ 1, let 

= Y^ieiP-^ ■ '^i be a (combined) strategy, by letting 77(7) = J2ieiPi ' ^o"^ 
all 7 in the domain. 

We overload the function S as from a state in S and a vector of strategies 
(of any depth n) tt G Tlf ^" x Tif x • • • x 7T|^" to X>(S'), by 5(s, tt) = S{s, a), 
where a{i) = Tr{i){s) for all i G E. Note each a{i) is a distribution over Actj. We 
further lift 5 to be a transition function from state distributions and strategy 
vectors to state distributions, by 



Sometimes we omit G, if it is clear from the context. 



Probabilistic Executions 



We settle the nondeterminism in a probabilistic game structure by fixing the 
behaviours of all players represented as strategies. Let G = {S, sq, Act, S) be a 
PGS, define a probabilistic execution £ as in the form of {E, A,C^ ,5^), where 

— EC 5'+ is the set of finite plays starting form a state in the initial dis- 
tribution and compatible with 6^, i.e., SoSi...Sn G E ii sq e \A], and 
5^{so ■ ■ ■ Si){so ■ ■ ■ Si+i) > for all < i < n, 

— A G 'D{S) an (initial) distribution, 

— £^ is the labelling function defined as £^(e) = C{last{e)) for all e £ E, 

— 6^ : E ^ 'D{E) is a (deterministic) transition relation, satisfying for all 
e € E there exists a (level 1) strategy vector tt,,, such that 6^{e){e ■ t) = 
6{last{e),7re){t) if t G \6{last{e),7re)], and otherwise. 

A probabilistic execution of G can be uniquely determined by a strategy vector 
TT starting from a state distribution. Given A G ^{S), define £{Q,ir, A) as the 
probabilistic execution {E'^ , A, C'^ , S"") , with = {JselA] I ^ for the 

set of compatible finite plays, defined as £'^(e) = £{last{e)) for all e G E'^ , 
and ^'^(e) = 6{last{e),Tre) for all e G E'" , where 7re(i) = 7r(t)(e) for all i G S. 
Intuitively, a probabilistic execution resembles the notion of the same name 
proposed by Segala and Lynch [Seg95b,SL95], and in this case the strategies of 
the players altogether represent a single adversary of Segala and Lynch. 

4 Probabilistic Alternating-Time Temporal Logic 

In this section we introduce a probabilistic version of alternating-time temporal 

logic [AIIK02] , which focuses on the players ability to enforce a property with an 
expected probability. Let Prop be a nonempty set of propositions. Probabilistic 
alternating-time temporal logic (PATL) formulas [CL07] are defined as follows. 

</>:=pH</.|0iA(/.2 I 

:= 00 I '/'iU^''02 

where A C i7 is a set of players, txG {<, >, <, >}, k G N Li {oo}, p G Prop, and 
a G [0, 1]. We also write ■0iU 4>2 for ipi\5-°°ilj2 as 'unbounded until'. The symbols 
<f>, (f>i, (f>2 are state formulas, and ijj is a path formula. We omit the syntactic 
sugars in our definition, such as true = p V —^p and false = p /\ —>p for some 
p G Prop, 01 V 02 = ~'(~'0i A -'02) for state formulas. The path modality R 
can be expressed by U without introducing negations into path formulas, as we 
will show later in this section. One may also define O-'^tj} = false R-'^tp, and 
O-'^tp = true U-'^tf), where k G NU {oo}. The set of PATL formulas L are the set 
of state formulas as defined above. We have the semantics of the path formulas 
and the state formulas defined as follows. 

— p\= (j) iS G, p(0) \= where is a state formula, 

— pHO0 iffp(i)N0, 



— p\= (pilS-''(p2 iff there exists i < k such that p{j) |= for all < j < i and 

— G,s \= p iS p £ >C(s), 

— G,s \= <j>i A (j)2 iS g,s 1= and Q,s\= (j)2, 

— G,s \= ((A))'^" ip iff there exists a vector of strategies tt G ^?^(j4), such that for 
all vectors of strategies tt' € -ff(A) for players in A, we have -Pr-g(^u-7r',s)({p G 
g{TT U tt', s) I p 1= V'}) CXI a, 

where p is an infinite play in Q, a G [0, 1], (f), ^2 are state formulas, and V 
is a path formula. Equivalently, given S the state space of a probabilistic game 
structure Q, we write |(/>] iov {s & S \ s \= cj)} for all PATL (state) formulas 
(j). For Z\ e we write Zi |= <?!) iff [Zi] C Intuitively, ^?,s ^ 

describes the ability of players in A to cooperatively enforce V with probability 
at least a in s. 

The following lemma is directly from the PATL semantics. If a group of users 
A can enforce a linear-time temporal logic formula i]j to hold with probability at 
least a with strategies tt G n[A), then at the same time tt enforces the formula 
-^xj} to hold with probability at most 1 — a. To simplify the notation, we let '~' 
denote changes on directions of the symbols in {<,>,<,>}, e.g., symbol > for 
<, < for >, > for <, and < for >. 

Lemma 1. g,s^ ((A))'^"^ ijfg,s ^ ((yl))^i-«^V 

Proof, (sketch) For all tt G n{A) and tt' G n{A), s e S and tp a path formula, 
we have Prg(^„u7r',s){{p e g{TvLiTv',s) \ p |= ib}) + Prgi^^u7r',s){{p e t/(7r U 
7r',s) I p 1= ^tp}) = 1. Therefore Prg(^^u-K',s)i{p_^ ^(tt U 7r',s) \ p \= ip}) a 
iff f rg(^u7r',s)({/3 € ^(tt U tt', s) | p |= -■■^})ixi1 - a, for all a G [0,1] and 
cxie {<,>,<,>}. □ 

Therefore, the path quantifi(;r R (release) can be expressed by the existing PATL 
syntax, in the way that ((A))^"(/)iR^ ^2 = ((^))'^^""(-'(/>i)U^'=(-.02), where both 
-■(/ii and -1(^2 are state formulas. 

On Model Checking of PATL 

In this section we briefly survey the results in the literature related to PATL 
model checking. Given a PATL formula in the form of {{A))'^°'ip{(f)i, . . . ,(j)n), 
regarding , . . . , as the sets of states satisfying state formulas 0i , . . . , 
a standard way to solve this problem is to determine the maximal or minimal 
probability that the players in A can enforce the LTL formula ijj{(f)i, . . . , </>„). In 
the following we write ip for ip{4'ij • • • i 4>n) without further confusions. 

LTL properties are special cases of w-regular winning objectives [Tho91] in 
two-player concurrent (zero-sum) games [dAM04,CdAH06]. In such games one 
may group a set of players ACS into a single protagonist and A into a single 
antagonist. Given an w-regular winning objective ^ and starting from a state 
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Fig. 1. An example showing tliat player I can guarantee to satisfy <>(j> with probability 
a for all < a < 1, but he cannot ensure that property with probability 1. 

s G S, the protagonist plays with a strategy trying to maximize the probabil- 
ity for a play to satisfy ^ while the antagonist tries to minimize the probabil- 
ity. In such a game there always exists a unique value in [0, 1], on which both 
players have strategies to guarantee (or infinitely approach) their best perfor- 
mances, regardless of the strategies played by their opponents. Such a supre- 
mum value (or infinum value, as for the antagonist) is called the value of the 
game [Mar98,dAM04]. In a probabilistic multi-player game, we let a group of 
players A C E he a single player, and A be the other, and the supremal proba- 
bility for A to enforce such an LTL formula tp starting from a given state s G S 
can be uniquely determined. This value is defined as 

{A)^{s)= □ n Prg(^u.'..)({pee(7rU7r',s) Ip^V}) 

Example 1. Fig. 1 gives a PGS with two players {I, II}, initial state sq, Acti = 
{01,02} and Actn = {&i,&2}- Note that this PGS is deterministic, i.e, no prob- 
abilities in its transitions. We assume that the only available transitions from 
Si and S2 are self-loops, and the other transition relations are as depicted in 
the graph. Suppose player I wants to maximize the probability to enforce the 
property 00, and player II aims to minimize it. 

Since the strategies applied on si and S2 do not matter, we focus on the 
choices of actions from both players on sq. We first focus on memoryless strate- 
gies, and let player I's strategy tti gives 7ri(7)(ai) = p and 7ri(7)(a2) = I— p 
for all 7 G S*"*". Similarly we let II assign probability q to 61 and 1 — g to 62 all 
the time. This produces an infinite tree, on which we write Xs„{l) for the actual 
probability I can achieve Ocj) from Sq, given the above memoryless strategies. 
(Note that a;si(I) = and Xg^i'^) = 1 in all cases.) This establishes an equation 
which further derives Xa„{l) = ^^^ij-py^pq^"^ ■ ^ simple analysis shows that when 
p approaches 1, the minimal value of Xso{l) approaches 1 as well, for all choices 
of q. That is, there exists a strategy for player I to enforce 0<p with probability 
1 — e for all e > 0. However, if player I chooses p = 1, player n may set g = 
so that a play will be trapped in sq for ever that yields Xso{l) = 0. The result 
of [dAM04] shows that in this case player I cannot do better even with general 



(history dependent) strategies. In fact there are no strategies for player I to 
enforce with probabiUty 1. □ 

Indeed, (A)V'(s) can be almost the best, i.e., we have G,s \= {{A}}- <^>'/'(«)-^^ 
for all e > [dAHK98]. Nevertheless, the quantitative version of determi- 
nacy [Mar98] ensures that for all LTL formulas ip and s G S, we have 

{A)^is) + (A)^^P{s) = 1 

The PATL model checking problems can be solved by calculating the values 
{A)il>s{s) for each state s, where each local objective Vs related to s might 
be distinct. The algorithms of [dAM04] define monotonic functions of type 
{S — >■ [0,1]) — (5 — [0,1]) to arbitrarily approach a vector {{A)tps{s)}g^s 
in a game structure with finite state space S with respect to an w-regular win- 
ning objective tp. Within each step one has to go through C'(|5'|) matrix games, 
and each iteration produces a unique fixed point. The algorithms on safety and 
reachability objectives are special cases of solving stochastic games [RF91]. More 
complex properties can be expressed as nested fixed points [dAM04] . Therefore, 
the upper bound complexities become exponential to the size of the winning 
objectives translated from LTL formulas. More recently, alternative algorithms 
proposed in [CdAH06] prove that for quantitative games with w-regular winning 
objectives expressed as parity conditions, whether the values of a game is within 
[r — e, r + e] can be decided in NP n coNP for all rational r £ [0, 1] and e > 0, 
which improves the theoretical upper bound for estimating the optimal values. 

Optimal Strategies 

It has been shown in [dAM04] that for safety games there always exist optimal 
strategies for the protagonists, however for reachability games it is not always 
the case. As shown in example 1, player I has no optimal strategy to enforce 
0(t> with probability 1 on sq even though (l)(^(so) = 1- Based on similar proof 
strategics applied in [dAM04] , we examine the existence of optimal strategies on 
winning objectives expressed as path formulas of PATL on a state. 

Lemma 2. Let s be a state, ip be a path formula, and A the set of protagonists. 

1. Ifip is of the form Qcj), (f)iU^''(j)2, (pi^-^cp^, or (/>iR(/)2 withk G N, there always 
exists a joint optimal strategy for A that enforces ip on s with probability at 

least {A)'ip{s). 

2. If is of the form <piU(p2, there always exists a joint e-optimal strategy for 
A that enforces tp on s with probability at least {A)'tp{s) — e, for all e > 0. 

For the prove of Lemma 2 we rely on the representation of a solution for a winning 
objective in quantitative game /i-calculus [dAM04]. For the sake of readability 
we leave the whole proof in the appendix. 

The next result proves the existence of a joint A strategy to enforce an PATL 

path formula with probability greater than a if there exists a joint strategy 
to enforce that formula with probability greater than a against an optimal A 
strategy. 



Lemma 3. Let be a PATL path formula and it' he a joint optimal strategy for 
the antagonists A on state s, if there exists a joint strategy tt for the protagonists 
A such that Prg(^u7r',s)({P e 5(7r Uw',s) \ p\= tjj}) > a, then g,s\= ((A))>"V- 

Proof Since tt' is the optimal strategy for the antagonists, we have for all joint 
strategies tt" , Prg(^„„u7T',s){{p S G{tt" L)tt',s) \ p \= tp}) < {A)tp{s), then we 
have {A)'ip{s) > a. If there exists an optimal joint strategy for A then we have 
s 1= ((^))-^'^^'^'(*'V-'i which implies s \= Otherwise by Lemma 2 there 

exists an e-optimal joint strategy for A with small e > to enforce ^ with 
probability at least {A)tp{s) — e> a. This also gives us s |= {{A))^°'ip. □ 

This result does not hold if we replace the operator ">" by ">" for unbounded 

until U. This is because if there does not exist a joint optimal strategy for A to 
enforce ^iU(/>2 with probability > a, we have no space to insert a tiny e > as we 
did in the above proof. For the fragment of path formulas without unbounded 
until, we extend the results for >, by the fact that optimal joint strategies for A 
always exist for these path modalities, as shown by Lemma 2. 

Lemma 4. For path formulas tp in the form of Q<j) or (j>iU^''(f)2 and optimal 
strategies tt' for the antagonists A on state s, if there exists a joint strategy tt 
for the protagonists A such that -Prg(^u^' € g{TT U 7r',s) | p \= ^}) ixi a, 
then g,s\= {{A)}^"^!, where A; € N and cxie {>, >}. 

Proof Since there exists joint strategies for A against ^'s optimal strategies, 
we have {A)-'ip{s)t><il — a, therefore {A)'ip[s) ixi a by determinacy. By Lemma 2 
there always exist optimal strategies for A to enforce tp with probability ixi a if 
tp is in the form of Q)(p or (^iU-'^02- □ 

A-PATL 

We define a sublogic of PATL by focusing on a particular set of players. Similar 
to the approach of [AHKV98], we only allow negations to appear on the level 
of propositions. Let ^ C i7, an ^-PATL formula is a state formula defined 

as follows: 

cP:=p\^p\<PiA<P2\<Pi^cP2\ {{A')r" O <^ I ((^T"<^iU-V2 I ((A'))>"<^iU(/.2 

where fc S N, ixie {>, >} and A' C A. Write for the set of ^-PATL formulas. 
An A-PATL formula describes a property that players in A are able to ensure 
with a minimal expectation by their joint strategies. Note that we only allow 
'> a' in the construction of unbounded until. 



5 Probabilistic Alternating Simulation Relations 

We define probabilistic versions of alternating simulation [AHKV98]. An alter- 
nating simulation is a two-step simulation. For a sketch, suppose state s is sim- 
ulated by state t. In the first step the protagonists choose their actions on t 



to simulate the behaviour of the protagonists on s, and in the second step the 
antagonists choose actions on s to respond the behaviour of the antagonists on t. 
This somehow results in a simulation-like relation, so that for a certain property 
the protagonists can enforce on s, they can also enforce it on t. To this end we 
split S into two groups of players — one group of protagonist and the other 
group of antagonist. Subsequently, we consider only the two-player case in a 
probabilistic game structure — player I for the protagonist and player n for the 
antagonist, since what we can achieve in the two-player case naturally extends to 
a result in systems with two complementary sets of players, i.e., AU A — S. For 
readability we also write the transition functions as d{s,ai,a2) and S{s,ni,TT2) 
for i5(s, (01,02)) and S{s, (711,112)), respectively. 

Let S, T b_e two sets andUCSxThea relation, then ncV{S)x V(T) is 
defined by ATZ0 if there exists a weight function w : S x T ^ [0,1] satisfying 

- X]teT^(*'^) ~ ^(*) for all s e 5, 

- J2ses ^) = ^(t) all t e T, 

- sTlt for all s e S and t€T with w{s, t) > 0. 

Note in this definition, it is equivalent to have X^tef©] w(s,t) = A{s) for all 
s € S, and ^g^^^-^ w{s,t) = 0{t) for all t € T. Since w can only assign non-zero 
values to the states in the support of A or 0. If w{s, t) > for some s ^ \A] and 
t G T, then we would have X^tg7'w(s,t) > = A{s), which is a contradiction. 
The followings are several properties of lifted relations. 

Lemma 5. (inverse) Let TZ~^ C T x S be the inverse of R C S x T, then for 

all A € V{S) and O € V{T), AUO iff&R^A. 

Proof. By taking the inverse of the weight function. □ 
Lemma 6. Let A € V{S), A' € V{S'), and U a relation on S. If AUA' , then 

1. If there exist Ai,A2, • • • € 'DiS) and an index set {pi}i satisfying J2ieiPi ~ 
1 and A = J2ieiPi ' ^'^^^ there exist A[,A'2 • • • G 'D{S') such that A' = 
J2ieiPi ' ^iR-^'i for all i e /. 

2. If there exist A\, A'2, - ■ • G 'D{S') and an index set {pi\i satisfying J2ieiPi ~ 
1 and A' = ^^^jPi ■ A'^, then there exist Ai, A2 ■ ■ ■ € I^{S) such that A = 
J2ieiPi ' "^i' ^^'^ AiR,A[ for all i G I. 

Proof. We prove the second part, and the first part is similar. Let A' = "^i^jPi- 
A'^, then define Ai for each i e / by Ai{s) = J2s'eS' '"'('^' •®') ' /^'(s') ^'^^ s G S. 
Now we can check that J2ieiPi ' ^i(^) = ^(*) for all s, i.e., A = J2ii£iPi ' 

To show that AilZA[, we define a weight function Wi : S x S' ^ [0, 1] by 
for all s G 5 and s' G 5", Wj(s,s') = w{s,s') ■ . Consider the following 

conditions. 

1. Wi{s,s') > implies w{s,s') > 0, therefore sTZs' . 

2. For all s e 5, we have Y^s'eS' «') = E^'eS' '^i^' ' 2^ = '^i(^)' 



3. For all s' £ S', we have 
A'{s') = A',{s'). 

□ 

Lemma 7. Lei TZ he a relation on S and {pi}i^i be an index set satisfying 
J2ieiPi ~ '^'^^ ^iT^^'i for distributions Z\j, A'^ e 2^(5') for all i, then J2ieiPi ' 
^iT^Eiei Pi -hi- 
proof W.l.o.g., let Ai_e V{S) and A'^ e V{S') for all i, and let Wi be the 
weight function for AiJZA^. Define a new weight function w : S x S' ^ [0, 1], by 
w{s,s') = J2teiPt ■ Wi{s,s'). 

— w{s,s') > 0, then ^ifrjPi ■ Wi{s,s') > 0, i.e, there exists some i G I such 
that Wi{s, s'} > 0, which gives sTZs' . 

— For all s € S, J2s'eS' "^(^^ 
= Es'eS' T,^eIP^■Ms,s') 

— To show that for all s' G S' , EseS ^(•^' ~ Eie/ft ' ^ii^') is similar. 

This gives Eie/ Pi ■ A^Eie/ Pi • ^i- ° 

Based on the notion of lifting, wc define the probabilistic alternating sim- 
ulation relation for player I that extends the alternating simulation relation 
of [AHKV98]. The definition for player n can be made in a similar way. 

Definition 1. Consider Q,Q' as two probabilistic game structures. A probabilis- 
tic alternating l-simulation QC S x S' is a relation satisfying if s C s' , then 

— £(s) = C'{s'), 

— for all TTi €E Uj'^ , there exists tt[ € ilf ''^ , such that for all € iln , there 
exists TT2 G n^'^ , such that (5(s, tti, 7r2)C d'{s' ,tt'i,tt'2) . 

Let TZ C S X S' and TZ' C S' x S" be two relations, then TZ-TZ' \s & relation on 
S X S" defined by s(7^ • TZ') s" if there exists s' e S' such that sTZs' and s'TZ's". 

Lemma 8. (Transitivity of alternating simulation) Consider G,G' and Q" be 
three probabilistic game structures. If Q C S x S' and C' C 5' x S" are prob- 
abilistic alternating 1- simulations, then Q ■ is a probabilistic alternating 1- 

simulation on S x S" . 

Proof, (sketch) Let s E • E' s", then by definition there exists s' & S' such 
that s C s' and s' C' s". Therefore C{s) = C{s') = C{s"). Let tti e iTf'\ 
then by definition there exists ttJ e ilf such that for all G /T^ there 
exists TTs G TTn'^ such that (tti, 7r2))C'^(.s', (tt^, tt^)). By s' C' .s", there 
exists tt" € ilf such that for all TTg G il^ there exists 773 G iln 
such that 5'(s', (7ri,7r3))C'(5"(s", (ttiiTTs)). Then from above there also exists 



TTs G Jlf'^ such that (5(s, (7ri^3))E<5'(s', «, tt^)). Write A = (5(5 , (tti, 77 3)), 
A' = 6'{s', «, 77^)) and A" = 5"{s", «, tt^)). we need to show that AQ ■ pz\". 

Let wi be a weight function for Z\CZ\' and u>2 a weight function for A'Q'A", 
define a new weight function w : SxS" [0, 1], by w{s, s") = J2s'eS' 
Let s e 5 and s" G S". 

— If w{s, s") > then exists s' G \A'] such that wi{s, s') > and ^2(5', s") > 
0, which imphes s C s' and s' C' s". Therefore, s C • C' s". 

EY~> Wi{s,s')-W2(3' ,s") 

sesZ^s'eS' A'(s') 

= Es'G5' "^2(5',^") 

= A"{s") 

— Showing Es"£S" w{s,s") = /i(s) is similar. 

□ 

Lemma 8 can also be derived from the transitivity of probabilistic alternating 
forward simulation (Corollary 1) and the fact that every probabilistic alternating 
simulation is also a probabilistic alternating forward simulation (Lemma 11). 

Based on the probabilistic forward simulation of Segala [Seg95a], and the 
alternating simulation of Alur et al. [AHKV98] , we propose the notion of proba- 
bilistic alternating forward simulation. A forward simulation relates a state to a 
distribution of states, which requires a different way of lifting. Let TZ C Sx 25(5) 
be a relation, write TZ for the smallest relation satisfying ATZ0 if there exists an 
index set {pi}iei satisfying Si^iPi = 1, such that A = Ei^jpi-sl, O = Si^ipi-Oi 
and SilZ&i for all i. We call TZ the forward lifting of TZ. Forward lifting has the 
following similar properties as the previous lifting. 

Lemma 9. Let TZ he a relation on SxV{S) and {pi}i^i be an index set satisfying 
^ieiPi ^ ^ '^^'^ AiTZA'^ for distributions Ai, A'^ G T^{S) for all i, then Eie/Pi ' 
AiTZ'^j^^j Pi ■ A!^, where TZ is the forward lifting of TZ. 

Lemma 10. Let A G V{S), A' G V{S'), and TZ a relation on S. If ATZA' , and 
there exist Z\i, Z\2, • • • G T^{S) and an index set {Pi}i satisfying J2ieiPi ~ ^^'^ 
A = J2ieiPi'-^i' *^6re exist A[, A'2--- G 'D{S') such that A' = '^i^jPi-A'^, 
and AiTZA\ for all i G I, where TZ is the forward lifting of TZ. 

Now wc define the probabilistic alternating forward simulation relation for player 
I, and the definition for player n can be made in a similar way. 

Definition 2. Consider two probabilistic game structures Q = {S, sq, C, Act, S) 
and Q' = {S',SQ,C',Act',S'). A probabilistic alternating forward 1-simulation 
CfC S X T>{S') is a relation satisfying if s Cf A', then 



- jC{s) = C'{s') for all s' G \A'}, 



{a) 



Fig. 2. An example showing that probabihstie alternating forward simulation is strictly 
weaker than probabilistic alternating simulation. 

— for all TTi G n^'^ , there exists tt[ G , such that for all tTj S , there 
exists 7r2 G n^'^ , such that 5(s, 7ri,7r2)Ef ^'(^',7''i5 7i'2)- 

Lemma 11. s Qt implies s Cf t. 

This lemma says that every probabilistic alternating simulation is a probabilis- 
tic forward simulation with a point distribution on the right hand side of the 
relation. The other way does not hold, i.e., probabilistic alternating forward 
simulation relates strictly more game structures than probabilistic alternating 
simulation. In Fig. 2, we assume Actj and Actn are both singleton sets. One 
may find that there are no states in the set {s'2, S3, S4, Sg} in Fig. 2(b) that can 
simulate states S3 and S5 in Fig. 2(a). Therefore, we cannot establish a prob- 
abilistic alternating simulation from si to s'^. However, si is related to s{ by 
probabilistic alternating forward simulation, since S3 (S5) can be related to a 
uniform distribution over S3 (s^ and s^). 

Before proceeding to the next step we introduce the following several auxil- 
iary lemmas. 

Lemma 12. Given {pi}ig/ with ^^^jPi = 1, s G S , {iTiJi^i C U^'^ and w G 
n^'^, we have S{s,J2ieiPi ' '^u'^) = J2ieiPi • ^(s, tt,, tt). 

Proof. Let t G S, then we have 

^(«'Eie/Pi • 7ri,7r)(t)= S{s,J2^^jPi ■ 7ri(s),7r(s))(t) 

= EaiGActi EaseActs Eie/ft ' M^i) ' ^^2) ' S{s, ai,a2){t) 
= T,ielPi ■ [EaieActi Ea2eAct2'^'(«l) ' ' ^(s, fli, a2)(t)] 

= Eje/P» ■ Hs,TT,is),Tr{s)){t) 

□ 



Lemma 13. Let Q and Q' he two game structures, {Z\j}jg/ he a set of distri- 

buttons, {ni}i^i a set of level 1 1-strategies, n G 11^ ' and {pi}i^i satisfies 

J2i<^iPi = 1; ™e haveS{J2.,^iPi-^i,'^',TT) = J2ieiPi'^i^i^'^i^'^)' where w' is a 
level 1 I strategy defined hy 

■ Ai{s) ■ TTiis) 



Proof Write LHS = <5(E^e/P. ' A,7t',tt) and RHS = E^eIP^ ' S{A,7r„n), 
we need to show for all t G S, LHS{t) = RHS{t). Write A for the distribution 
^^gjPi • Ai. Let t G S, then by definition 

LHS{t)= E.er^l E^eIP^ ■ A{s)~Sis,Tr',n)it) 
= J:se\A^As)■S{s,7T',n){t) 
= E.6r^l A{s)-S{s,7T'{s),n{s)){t) 

= E.er^l ^(s) • Ejei '-'^Hs,7rj{s),7T{sm By Lemma 12 

= Esef/il ■ Ej6/Pj • ^j(s) • ^(s,7fj(s),7r(s))(t) 
= EjeiPj ■ E.er^i -A-W • S{s,n,is),n{s)){t) 
= J2jeiPj ■ T>se[A,] -Ais) ■ S{s,nj{s),'K{s)){t) 

= '^j^jPj ■ S{Ai,'!Ti,Tr){t) By definition 

= RHS{t) 

□ 

The next result shows that the definition of forward simulation also works 
on the lifted relation. 

Lemma 14. If A\ZfO, then for allni G Hj'^, there exists tt2 G Uj , such that 
for all TTj G , there exists tt'^ £ n^'^ , such that 5{A, tti, 7rJ)Cf 6{0, 1x2, 1^2)- 

Proof. By definition there exists a set {pi\i£i such that A — '^^^jPi -Si, = 
J2ieiPi ' — f ^i- ""1 ^ ^1'^ ^® ^ (mixed) I-strategy. Then by 

definition, for alH e / there exists tt, e Iff such that for all tt- e there 
exists TT-' e n^'^ such that tti, 7r-')Cf J(si, tt^, tt-). Now we take ^2 defined 
by 7r2(s) = J2ieiPi ^(s) ^» ( ^ ) be the required (level 1) I-stratcgy. 

Let Tr'2 G TTn'^ 

, we prove as follows the existence of another Il-stratcgy 
tt[ e iln'^ t;hat satisfies (5(Z\, tti, 7ri)Cf ^(0, 7r2, 712). For each i G /, by Cf 
0i, there exists n'/ G U^'^ satisfying 5(si, tti, 7r")Cf(5(0^, 772, 772). Then we have 
Eie/ Pi •^(si,7ri,7r-)EfEiG/P»-'^(6'i, 772,772), by Lemma 7. The required (mixed) 

n-strategy 77^ is defined by 77^(3) = Eie/Pj3(fy^i'(*) fo'^ '"■^l ^' ^'^'^ the result 
follows from Lemma 13. □ 

Consequently, we are able to show that lifted probabilistic alternating forward 
simulations are transitive. 



Corollary 1. (Transitivity of alternating forward simulation) Let Cf be a prob- 
abilistic alternating forward 1- simulation, then A-iQfA2 and /i2Ef^3 implies 

6 Forward I- Simulation is Sound for I-PATL 

This section establishes the main result of the paper: a relationship between 
probabilistic forward I-simulation and I-PATL formulas. Recall that a I-PATL 
formula has only strategy modalities ((I)) and ((0)), and negations are only al- 
lowed to appciar immediately before the propositions. For readability we write 
((I)) for (({I})). Let g and Q' be two PGSs, A e V(S) and A' e V{S') such that 
AQfA' by a probabilistic alternating forward I-simulation. We need to show 
that A\=(l) impUes A' \= (f) for all I-PATL formula (/). 

Our proof relies on the existence of player II 's optimal strategies for path 
formulas as winning objectives (as shown in Sect. 4). Suppose tti is a I strategy 
that enforces 0, we construct another I strategy ttJ that simulates tt all along the 
way, in the sense that provided the optimal n strategy ttj there exists another H 
strategy such that the probabilistic execution £{0, {-^1,^2), A) will be "simu- 
lated" by the probabiHstic execution £{0', {7r[,Tr2), A'). Since tti enforces <j), the 
£{G, (tti, 7r2), A) satisfies (f>, we show that it is also the case of £{g' , (tti, TTg), A'). 

Let £ = {E, A, and £' = (£". A', C^\5^') be probabilistic executions 

of Q and Q' , respectively. Also let Cf C 5* x 2?(S") be a probabilistic alternating 
forward I-simulation. We say the pair {£, £') is an instance of simulation, by 
writing £ Q £' , if there exists a (simulation) relation C'C E x 'D{E'), such that 

- AH' A', 

- if e C' e then last{e) Cf last{0), 

- if e then (5^(e)C(5^'(6)), 

where last(0) is a distribution satisfying last{0){s) = Yliiast(e)=s^i^)- ^ 
properties of the relation Q' are as follows. 

Lemma 15. 1. AC^O implies 5^ [A)^ 5^' {Q) . 

2. A\Z and A = Ai(Ba ^2 with a € [0, 1], then there exist 0i,02 such that 
Air'0i, A2Q'02, and = 0^® ^02. 

A proof of part (1) is by definition of C' and Lemma 9, and part (2) holds by 
Lemma 10. 

Let Z\ be a state distribution of tj, Z\' be a state distribution of Q' , and 
A^fA'. Suppose TTi is a I strategy in Q that enforces (j) with probability at least 
a, and ttj is a n strategy in Q' , step-by-step we establish a I strategy tt[ and a IE 
strategy 112, so that the probabilistic cxecutution decided by tti and tt2 from A 
will be simulated by the probabilistic executution decided by tti and 772 from A'. 

Lemma 16. LetG = {S, so, Act, S) andQ' = {S' , s'q, C , Act' ,S') be two PGSs. 
If AQfA', then for all tti e ilf and ttj S , there exists tt^ G Ilf and 
■K2 e ng, such that £{g, {tti, Tr2), A) C £'{g' , {'K[,n'2), A'). 



Proof. We construct 7r\ and wl as a level 1 strategies of player I and II for 
all i e N, and define 77^(7 • s) = Tr^^^~^^{s) for all 7 e 5* and s e S. And 
712(7 ■ •'^) = T^2^^\s) for all 7 e (5")* and s G S". 

Since Z\Cf/l', then by Lemma 14, there exists tt} G ilf such that for all 
TT^' e 71^''^ there exists tt^" S iT^'^ such that tti, 7r^")Ef^(/i', tt^ tt^')- So if 
we take the first level of 772, there exists irl € U^'^, such that 6{A, tti, 7r2)Ef 7r| , 772). 
We define Zi2 € ViS"^) byZ\2(siS2) = zi(si)-J(si, tti, 7r^)(s2), and e V{{S'f) 
in a similar way. We also 'truncate' the strategy tti by defining 7ri(2) G ilf 
in the way that 7ri(2)(s) = J2s'e[A] ^i-''') ' 7ri(s's). And we define 7r2(2) in a 
similar way. 

Suppose we have Z\„,Z\^ e I?(S'"), and 7ri(n) e nf'^, and 7r2(s) e 77^ '""^j in 
the similar way to above, we construct tt" and 7r2 , such that S{A, 7ri(n), ttJ )Cf(5(zi', tt", 7r2(n)). 
Then we define Zi„+i G I?(S'"+^) by Z\„+i(si . . . s„s„-|-i) = /i„(si...s„) • 
5(s„, ^i(n), ^2")(s«+i), and Z\;,+i G 7?((y)«+i) by A'^,+,isi ■ ■ ■ ^'^n^n+i) - A'Jsi . . . s„)- 
(5«,7r5',7r2(n))(s„+i). Wc then define 7ri(n + 1) G ilf'^ by 7ri(r7, + l)(.s) = 
E^e\A„^ ^„(7)-7ri(7-s), and4(«+l) G hy n',(n+l){s) = EyelA'J ^nh)- 

Ail- s). 

It is easily verifiable that we have established two probabilistic executions 
satisfying £{Q, {tti,tt2),A) C £'{G' , {7t'i,tt'2) , A'), by taking a probabilistic alter- 
nating forward simulation as C'. □ 

In order to measure the probability of a path formula to be satisfied when 
the strategies from both player I and player n are fixed, we define a relation 
|_ixia £qj- probabilistic executions. 

Definition 3. Let Q be a probabilistic game structure, S{A) = {E, A, £^ , 6^) a 
probabilistic execution determined by a strategy vector -rz£, and if) a path formula, 
define 

£{A)^^'^i, iff Pri{{p€ y g{ize,s)\p^i^})>ia 

It is conceivable that in a probabilistic execution every finite or infinite trace 
in E* U E^ maps to a trace in Q, in the way tliat p = 616263 ... is a trace in 
£ implies that proj{p) = last{ei)last{e2)last{e:i) . . . is a play in Q, where the 
function proj projects every finite sequence of states in E into its last state in S. 
Consequently, we let Pr^ be a probabilistic measure over i?", such that for the 
cone sets (of finite traces), wc have Prf{e) ~ A{last{e)), and Prf{'-f ■ ci • 62) = 
Prg{j ■ Ci) • S^{ei){e2), for -f ^ E* and 61,62 G E. Let p be an infinite trace 
in £, we write p \= ijj iS proj(p) \= ip. Similarly, for a state formula (j) and 
e G E, write e G Icf)} iff last{e) G \4>\. In the following we study the properties of 
the satisfaction relation for a probabilistic execution to satisfy a I-PATL path 
formula by means of unfolding. 

Lemma 17. Letcp, and (1)2 be 1-PATL (state) formulas, andcxg {>,>} then 



1. £{A) \='^'^ 0<j) iff there exists a' [xi a, such that 5^ (A) = Ai ®a' ^2 with 
\Ai~\ n \A2] = 0, and Ai \= </>. 

2. £{A) \='^°' (j)iU^^(f)2 iff there exists a finite sequence of triples {((Z\i,Oj ai,o)) 
(A,i,ai,i), (A,2,Q:i,2))}o<i<j Sor some j < k, with [A,^l n [A,«'l = for 
all distinct (.' G {0, 1, 2} and < i < j, such that 

ie[o...j] \ i'e[o...i-i] / 

(2) A = J2ie{o,i,2} '='o,i ■ A04, and i5^(A,o) = J2ee{o,i.2} " A+i/ f^r 

alio < i < j, (3) Aifi \= </>! and A,i H ^2 for allQ < i < j. 

3. £{A) 1=1^" (p-^^u iff there exists a finite or infinite sequence of triples 

{{{Aifi,aifl),(Ai^i,ai^i),iAi^2,Q.i,2))}o<i<] for some j € N+ U {00}, with 
lA^^i] n \Ai^i,~\ = for all distinct £,£' e {0, 1, 2} and 0<i < j, such that 

(1) "^'1 ■ II 

o<i<j \ i'e[o...i-i] / 

(2) Z\ = J2ee{o,i,2} (^o,i ■ Aq^i, and (5^(A,o) = I]<!e{o,i,2} "i+i/ ' /or 
alio <i < j, (3) i\,,o 1= fl'^c^ A,i 1= (l>2 for allO <i < j. 

For readability we leave the proof of this lemma in the appendix. 

Theorem 1. Let G = {S,so,C,Act,d) and Q' = {S' , s'q, C , Act' ,6') be two 
PGSs, CfC Sx'D{S') a probabilistic alternating forward 1-simulation. If AQfA' , 
then G,A\=(j) implies Q' , A' \= (j) for all ^ e Lj. 

Proof, (sketch) Wc prove by induction on the structure of a I-PATL formula (j>. 
Base case: suppose A \= p, then s ^ p for all s € \A]. By A\—fA', there exists 
an index set {qi}iei satisfying J2ieiQi = 1, A = Y^^^jqiSi, A' = EjG/^iA, 
and s, Ef Ai. Therefore C{si) = £'{t) for all t G \A,]. So t \= p for all t G \Ai] 
for all i. Therefore A' \= p. The case of -ip is similar. 

We show the case when (p = ((I))->"(/)iU(;62, and the proof methods for the 
other PATL path constructors arc just similar. Since for all t <E \A'~\ there exists 
an optimal strategy tt* for the winning objective ->(piTZ->4>2 by Lemma 2(1), and 
we combine these strategies into a single strategy ttj satisfying w'2{t-a) = n*{t-a) 
for all t € \A'~\ and a € S* . Then ttj is optimal for ^0i7^^(/)2 on A'. Then by 
Lemma 16, there exist 1^2 G and tt^ G ilf such that £{Q, {1^1,^2), A) C 
£'{Q', (tti, 772), A). Since tti enforces (f)iV(p2 with probability greater than a, we 
have £{A) |=>" 0iU(/)2. 

Then by Lemma 17(3) there exists a finite or infinite sequence of triples 
{((A,o,Q!i,o), (A,i,Q!j,i), (A,2,ai,2))}o<i<j for some j e N+U{oo} satisfying the 
properties as stated in Lemma 17(3). By repetitively applying Lemma 15 we es- 
tablish another sequence of triples {{{A'^ Q,aifl), {A^ -^^,ai^l), (A 2, Q;i,2))}o<i<i5 
such that (1) Eo<i<jKi-ni'e[o...i-i] "i',o) > (2M' = E^e{o,i,2} "o/- A,^' 



and 5^{A[ q) = E<!6{o,i,2} ' ^^"^ all < i < j, (3) Z\i,o!=f^i,o and 

^i.iEf^i 1 for all < i < j. By induction hypothesis we have A[q \= <t)\_ and 
A'l^ 1= (t>2 for all < f < j. Therefore £{A') (^iU(^2 by Lerama' 17(3). 

Since is an optimal strategy of H, we have A' \= ((I))>"(/)iU02 by Lemma 3. 

For a formula {{^))^°''tl) we apply the same proof strategies as for {{l))'^"ip, 
except that player I does not need to enforce ip with a certain probability C><1 a 
since every probabilistic execution generated by a pair of I and H strategies will 
enforce ip with that probability. □ 



7 Probabilistic Alternating Bisimulation 

If a probabilistic alternating simulation is symmetric, we call it a probabilistic 
alternating bisimulation. 

Definition 4. Consider two probabilistic game structures Q = {S,so,jC,Act,S) 
and Q' = {S' , s'q, C , Act' ,S') . A probabilistic alternating 1 -bisimulation c^^C S x 
>S" is a symmetric relation satisfying if s s' , then 

-C{s)=C'{s'), 

— for all TTi € n^'^ , there exists 7r[ <£ 11 j , such that for all ix'^ € il^ , there 
exists TT2 G -ffn'^j such that (5(s, tti, 772)^ d'{s' ,tt'i,tt'2), 

where ^ is a lifting of ^ by weight functions. 

Since every probabilistic alternating I-simulation is also a probabilistic al- 
ternating forward I-simulation by treating the right hand side state as a point 
distribution (Lemma 11), the lifted probabilistic alternating I-simulation is also a 
lifted probabilistic alternating forward I-simulation. This fact extends for bisim- 
ulation. A probabilistic alternating I-bisimulation also preserves formulas in Lj. 
Moreover we write for the set of formulas defined as follows, which allows 
negations to appear anywhere in a formula, and further we are able to show that 
probabilistic alternating bisimulation preserves all properties expressed in L^. 

(l>:=p\^<P\^iA(f>2\ O </> I UrcpiV^'^h I {{A'y(l>rV(f>2 

Theorem 2. Let G = {S,so,jC,Act,6) and Q' = {S' , s'q, C , Act' , 5') be two 
PGSs, ~C S X S' is a probabilistic alternating 1-bisimulation. For all s G S 
and s' e S' with s ~ s' and (j) € Lj", we have G,s\= (j) iffQ', s' \= (j). 

The proof methodology basically follows that of Theorem 1, besides that when- 
ever Ac^A' and A \= ^(j>, we show that if there were s' G ' such that Q' , s' \= cj) 
then we would also have G,s \= (p for some s G \A'\, which is a contradiction. 
And from that we have A' \= -^cp as well. 



8 Conclusion and Future Work 



We report our first results on probabilistic alternating simulation relations. We 
have introduced two notions of simulation for probabilistic game structures - 
probabilistic alternating simulation and probabilistic alternating forward simu- 
lation, following the seminal works of Segala and Lynch [Seg95a,SL95] on proba- 
bilistic simulation relations and the work of Alur et al. [AHKV98] on alternating 
refinement relations for non-probabilistic game structures. Our main effort has 
been devoted to a logical characterization for probabilistic alternating simulation 
relations, by showing that they preserve a fragment of PATL formulas. 

On our way to the main result, we find that the proof strategy accommo- 
dated in [AHKV98] no longer applies, due to the failure in reconstructing a 
strategy from sub-strategies with the existence of probabilistic behaviors. Note 
that alternating simulations rely on mimicking behaviors by strategics of depth 
one, while enforcing a PATL property needs to fix a general strategy (of infinite 
depth) from one party regardless of any strategies of the other. We circumvent 
this problem by incorporating the results of probabilistic detcrminacy [Mar98] 
and the existence of optimal strategies [dAM04] in stochastic games. 

There are several ways to proceed. We want to study the completeness of log- 
ical characterization for probabilistic alternating forward simulation. It is also 
of our interest to investigate the complexity for checking probabilistic alternating 
simulation relations by studying the results in the literature [AHKV98,BEMC00] . 
Our work was partially motivated by the paper [ASW09] , where PATL is used to 
formalize a balanced property for a probabilistic contract signing protocol. Here, 
a balanced protocol means that a dishonest participant never has a strategy to 
unilaterally determine the outcome of the protocol. It is interesting to see how 
much the development of simulation relations for probabilistic game structures 
can help the verification of such kind of security protocols. 
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A A proof of Lemma 2 

The proof relies on the representation of a solution of an LTL path formula as a 
winning objective in quantitative game /x-calculus [dAM04] for a two-player (I 



and n) game. Its grammar is defined as follows. 



(/) := Q I a; I 01 V (^2 I (/>! A 02 I Pprei{<j)) \ Ppren((/>) | l^x.cj) \ vx.cj) 

The semantics of such formulas map each formula into J- , the function space 
5 — ^ [0, 1]. A member f & T gives an expected value f{s) for player I to win 
the game on every state s G S. There is a partial order defined on J" in the 
way that given two functions /, g G / < g if f{s) < g{s) for all s € S. 
For Q C S", it represents a function that Q{s) = 1 if s € Q and Q{s) = 
otherwise. For conjunction and disjunction, they are defined as (/ A g){s) = 
min{f{s), g(s)} for all s G S, and (f y g){s) = max{f{s), g{s)} for all s Cz S. 
The quantitative predecessor operator Pprej for player I and for every f G is 

by Pprej{f){s) = \J^^^n, H^.en^ Es-efsisA^u^?})] {^i^''2)){s')f{s') for all 
s Cz S. The operator Ppreji can be defined in a similar way. Intuitively, based on 
/, Pprei{f) gives the maximal expectation of player i on each state s after one 
move, and the existence of such max;imal strategy and values are guaranteed by 
the minimax theorem [vNM47]. Finally, fix.(f){x) = \~\{f G T \ (p{f) < /} and 

ux4ix) = U{/ € ^ I Hf) > /}• 

The existence of the optimal strategy for player I on an LTL objective can 
be sketched as follows. 

— For 0<A) we construct the optimal strategy from Pprei{(j)) by solving a ma- 
trix game on each state s G S' on reaching states in 10]. In this case we 
only need to construct a level 1 strategy on every state, with its existence 
guaranteed by the minimax theorem. 

— For bounded until 0iU-'^02, we do the following construction recursively 
and prove the property by induction. 0iU-°02 = 102] works for every strat- 
egy in a state in [02]. For A; > 0, we interpret 0iU-'°02 as 02 V (0i A 
Pprei(0iU^'=-i02))- Then suppose there exists an optimal strategy for 0iU- 
we only need to prolong the optimal strategy by one additional level, based 
on the expected value already computed for 0iU-*^~^02. 

— The case of bounded release 0iR-'^02 it can be shown in a similar way as 
the above case, by letting 0iR-°02 as |02], and 0iR-'°02 as 02 A (0i V 
Pprei(0iR-*=~^02)) for each fc > 0. 

— For unboimdcd release 0iR02, our argument resembles the proof of [dAM04, 
Lemma 2] on safety games. The value of the game for player I as the protag- 
onist is interpreted as the function / ~ j^x.02A(0i VPprei(a;)), and there ex- 
ists a memoryless strategy tti G U^'^ for player I so that on each state s G S, 
TTi{s) G r'(Acti) is the best choice (in the matrix game on s) player I can 
make according to the greatest fixed point /, i.e., for all memoryless player H 
strategies 7r2 G il^'^, we have X^s'es '^(■'i '"'Ij ''''2)(s') ' /(■*') > f{s). Wc show 
that TTi is the strategy that guarantees /(s) on each state s in the general 
sense. Let 7r2 G be an arbitrary player n strategy, and s e 5, we are go- 
ing to show that in the probabilistic execution £{G , tti U 7r2, s) = {E, s, C, 5), 
we have £{s) |=-/(*) 0iR02- In order to do so, we give the following inter- 
mediate result that £{s) |=^/(*) 0iR"02 for all n e N. 



We prove this by induction on n G N that £{e) ^>/('««*(e)) 0iR<"(^2 for all 
e E E. By abuse of the notation we treat tti also as a general strategy such 
that 7ri(7) = TTi{last{'^)) for all 7 e 5+. Wc also write 7r| as the "truncated" 
strategy of n2, by defining 7r|(s -7) = 7r2(e • 7) for all s e 5 and 7 € 5+. To 
simphfy notation we write Pr|(7ri, 772, V') for € ^((7ri,7r2), last{e)) \ 

Base case: let e e S, n = and (/)iR-°02 = </^'2, we have Pr|(7ri, 7r|, (/)iR-°02) = 
1 > f{last{e)) = 1 if /asi(e) e |?;i2l, and Pr|(7ri,7r|,(^iR^°02) = > 
f{last{e)) = otherwise. 

Suppose this holds up to level n, we need to show the case of n + 1. 

• If last{e) G |^(/)2l, then Pr|(7ri, 7r|, ^iR^"+V2) = > ,f{last{e)). 

• If «asi(e) G |(/)2l n then Pr|(7ri, 7r|, 0iR^"+V2) = 1 > f{last{e)). 

• lilast{e) G Mnh^il,thenPr|(7ri,7r|,</.iR^"+V2) = Ee'6[5(e)l '5(e)(e')- 
Pr|'(7ri,3^',<^iR^"02). By I.H., Pr|'(7ri, 7r|', <^iR^"02) > f{last{e')), we 
have Pr|(^i,^|,<^iR^"+V2) > Ee'er5(e)l '^(e)(e') • f{last{e')). 

By definition wc have f{last{e)) 

< E.'e5(!a.t(e),,ri,^.)^(^«s*(e),7ri,7r2)(s')/(s') 

= Ee'er%)l'^(^)(^')-/(^«st(e')). 

The last equivalence is by definition of £{G,tti U 7r2,s). The result im- 
mediately follows. 

Pr| (tti , 7r2 , (i)iR(/)2 ) = lim„^co Pr's (tti , 7r2 , 0i R" (/)2 ) , we have Pr| (tti , 7r2 , R(/'2 ) > 
/(s). Since 7r2 is arbitrarily chosen, we have tti is a player I strategy that 
enforces ^iR02 with probability at least f{s). Moreover, since s is arbitrarily 
chosen, tti is optimal on s for all s G S. 

The existence of e-optimal strategies for (/'iU(/i2 for all e > is guaranteed by the 
existence of e-optimal strategies for all w-regular winning objectives, as shown 
in [dAM04]. 

B A proof sketch of Lemma 17 

1. Define the set {s G [(5^(Z\)] | s \= (j)}. Now it is obvious that for all infinite 
run p G -E", p \= Qcf) iff p(l) |= 0. The result immediately follows. 

2. The proof this item is similar to the below in its finite case. 

3. Given the probabilistic execution £{A), intuitively, each e G E represents 
a finite run within £{A). Wc construct as follows a maximal sequence of 
triples {((A,o, a^.o), (A,i, {A,2, ^1,2))} for i G N. 

For all e G E, we define Ei^i = G E \ e = S1S2 • • • Sj; |= (p2,Sj \= 
(bi for all j < i}, and Ei^Q = {e E \ e = S1S2 ■ ■ ■ s-i, Sj \= ipi for all j < i}. 
Intuitively, Ei^i are the prefixes of those runs that satisfy (pi\J(j)2, and Ei^i 
are the prefixes of the runs that might satisfy (piV(f)2- 

Further we define Aq = A and £"9,2 = [^ol \ (Eq.o U Eq^i), and ao/ = 
Zio(-E'o,^) for ^ G {0,1,2}. Then for each i G N, recursively define = 
S{Aifi), Ei+i^2 = [A+i] \ (Ei+iflU Ei+i^i), ai+i,i = Ai+i{Ei+i^e) for all 



£ G {0, 1,2}. Consequently, we have Ai^i{e) = Ai{e)/ai^c if e e Ei^^ and 
otherwise, for aU i G N and £ e {0, 1, 2}, provided aj_^ ^ 0. If ai^i = we let 
Ai^^ be empty, i.e., it assigns every e € ii^ to 0. 

It is easily verifiable that \A^^i \ n \A^^i,'] = for all distinct e {0, 1, 2}, 
since Ei^^nEi^i' = 0. Also for all i G N we have (2) S^{Aifi) = J2ee{o,i,2} 0^+1,1 
Z\j-i-i_£, and (3) Z\i.o |= 0i and Z\j_2 |= (f'^ for all i. 

For every infinite run p G i?", wc have p |= (t)il!4'2 iff there exists a prefix 
e G ii'i.i for some i gN. Therefore we have Priiip e E'^ \ p ^ (j)iV(f,2}) 
iff Y^i(zf^{cti.i'Y[o<j<i '^j-o) fo^ {>: which is (1). That is, the collection 
of infinite traces satisfying (/)iU02 arc those with prefix cqCi . . . with |= 02 
and ej ^ 0i for all < j < i- Therefore, suppose £{A) \='^ (/)iU02, we have 
that the above sequence of triples {((A,o, ai,o), (A,i,Q:i,i), (Zii,2, aj,2))}ieN 
satisfies the required conditions (1), (2) and (3). 

Suppose there exists a sequence of triples satisfying (1), (2) and (3) with 
respect to [XI a, due to a similar way of reasoning, we already collected 
enough infinite runs that satisfy (pi'U(p2 with probability cxi a for cxie {>, >}. 



